My D1 DB running the new experimental backend has stopped working entirely, any clue on why? All Que

M

Mozzy•7/6/23, 2:18 PM

How does that work with data consistency? Let's say I write to it, and I want the user in any location to immediately be able to get that fresh write?

Can I opt-in to / choose this in any way?

MMozzy How does that work with data consistency? Let's say I write to it, and I want th...

M

Matt Silverlock•7/6/23, 2:29 PM

We’ll manage that behind the scenes so you can read your writes. When we ship replication there will be docs covering all of this

CChaika My D1 DB running the new experimental backend has stopped working entirely, any ...

M

Matt Silverlock•7/6/23, 2:44 PM

Looking. Is this still occurring?

C

ChaikaOP•7/6/23, 2:46 PM

Thanks! Yea it seems to be in an even worse state now, it's not instantly erroring, but infinitely loading, queries against it, and trying to load it in the dashboard, just take so long I get a gateway timeout
Edit: Looks like I'm back to instant d1 errors, same error as above. Still broken sadly

MMatt Silverlock We’ll manage that behind the scenes so you can read your writes. When we ship re...

M

Mozzy•7/6/23, 2:49 PM

Okay thanks

CChaika Thanks! Yea it seems to be in an even worse state now, it's not instantly errori...

M

Matt Silverlock•7/6/23, 10:03 PM

Still broken?

A

Andrezzz•7/6/23, 10:09 PM

hey guys, can we manage the database using some client?

AAndrezzz hey guys, can we manage the database using some client?

M

Matt Silverlock•7/6/23, 10:17 PM

The Drizzle team are working on support in Drizzle Studio - https://twitter.com/drizzleorm/status/1675617777636212738?s=46

Drizzle ORM (@DrizzleOrm)

We will make @Cloudflare D1 adapter this week for Drizzle Studio, should be great

Likes

105

Twitter

•

7/2/23, 9:29 PM

MMatt Silverlock The Drizzle team are working on support in Drizzle Studio - https://twitter.com...

A

Andrezzz•7/6/23, 10:17 PM

thanks

A

Andrezzz•7/6/23, 10:17 PM

I'll dump it and upload it

MMatt Silverlock Still broken?

C

ChaikaOP•7/6/23, 10:53 PM

it is yea
972558fd-cadf-4bec-8b0b-5a140d7d51fb

C

cris•7/6/23, 11:42 PM

@elithrar can I jump on this thread or should I open a new one? I'm getting an error when trying out your tutorial.

Unexpected token

Unexpected token

Unexpected token

Unexpected token

Ccris @elithrar can I jump on this thread or should I open a new one? I'm getting an ...

M

Matt Silverlock•7/6/23, 11:45 PM

This is definitely not related to Chaika’s issue. I think you have a typo in your code (unexpected token).

MMatt Silverlock This is _definitely_ not related to Chaika’s issue. I think you have a typo in y...

C

cris•7/6/23, 11:54 PM

It wasn't immediately clear to me that this is a subchannel for all things related to d1-open-alpha -- sometimes I overlook the obvious.

C

cris•7/6/23, 11:55 PM

But the error I got, if it's a typo, I'm pretty sure I copied and pasted from the example you give on Github.

Ccris But the error I got, if it's a typo, I'm pretty sure I copied and pasted from th...

M

Matt Silverlock•7/6/23, 11:59 PM

Which example? Mind stepping back a sec and helping me understand a) what you’re trying to run / b) the code?

MMatt Silverlock Which example? Mind stepping back a sec and helping me understand a) what you’re...

C

cris•7/7/23, 12:02 AM

Thanks, I'm trying to run the https://github.com/elithrar/svelte-demo-d1 and I put the sample code in src/routes/api/users/+server.js -- (the only difference I see is that I'm using .js and not .ts)

GitHub

GitHub - elithrar/svelte-demo-d1: Cloudflare Pages + SvelteKit + Cl...

Cloudflare Pages + SvelteKit + Cloudflare D1 - https://developers.cloudflare.com/pages/framework-guides/deploy-a-svelte-site/#deploy-with-cloudflare-pages

GitHub - elithrar/svelte-demo-d1: Cloudf...

Ccris Thanks, I'm trying to run the https://github.com/elithrar/svelte-demo-d1 and I p...

M

Matt Silverlock•7/7/23, 12:07 AM

If you aren’t building TypeScript I can’t guarantee it will work. That is also my personal account - not an officially supported project. I strongly recommend using our official tutorial first: https://developers.cloudflare.com/pages/framework-guides/deploy-a-svelte-site/#deploy-with-cloudflare-pages

Deploy a Svelte site · Cloudflare Pages docs

Svelte is an increasingly popular, open-source framework for building user interfaces and web applications. Unlike most frameworks, Svelte is …

Ccris Thanks, I'm trying to run the https://github.com/elithrar/svelte-demo-d1 and I p...

M

Matt Silverlock•7/7/23, 12:11 AM

I have that exact repo deployed as a project: if you're changing things it's probably breaking it.

C

cris•7/7/23, 12:11 AM

I have deployed several sites already, following both the Cloudflare tutorial and Sveltekit's guide. Functions are proving a challenge.

M

Matt Silverlock•7/7/23, 12:12 AM

https://svelte-demo-d1.pages.dev/api/users -> returns results.

M

Matt Silverlock•7/7/23, 12:12 AM

If this is specific to Pages Functions, I'd ask over in #pages-help

MMatt Silverlock If this is specific to Pages Functions, I'd ask over in #pages-help

C

cris•7/7/23, 12:14 AM

I have started reaching out there, too.

M

Matt Silverlock•7/7/23, 12:14 AM

I just forced a rebuild of that Pages site — unchanged, not a line changed from my original push ~3 weeks ago — and it works: https://32b48f37.svelte-demo-d1.pages.dev/api/users latest deploy, just pushed ~10 seconds ago.

I would ask in #pages-help and be very clear about what you're changing here.

MMatt Silverlock I just forced a rebuild of that Pages site — unchanged, not a line changed from ...

C

cris•7/7/23, 12:16 AM

I'll keep at it. What does your users database look like, do you have the schema?

C

cris•7/7/23, 12:18 AM

I see it from the output, email, id, etc. but I haven't actually added any tables to the db. I was hoping it would just return some system info to start. Maybe the error has to do with an incomplete db setup.

Ccris I'll keep at it. What does your users database look like, do you have the schem...

M

Matt Silverlock•7/7/23, 12:20 AM

It's impossible to tell from the error you've posted. If you haven't populated your DB - it won't work.

The query in the example app is just

"SELECT * FROM users LIMIT 5"

"SELECT * FROM users LIMIT 5"

"SELECT * FROM users LIMIT 5"

"SELECT * FROM users LIMIT 5" — it doesn't matter what the schema is because the query doesn't do anything smart.

➜  wrangler d1 execute db-enam --command "PRAGMA table_xinfo(users)"

🌀 Mapping SQL input into an array of statements
🌀 Parsing 1 statements
✔ Select an account › a single plum floating in perfume
🌀 Executing on db-enam (c89db32e-83f4-4e62-8cd7-7c8f97659029):
🚣 Executed 1 commands in 0.20019398629665375ms
┌─────┬───────────────┬─────────┬─────────┬────────────┬────┬────────┐
│ cid │ name          │ type    │ notnull │ dflt_value │ pk │ hidden │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 0   │ user_id       │ INTEGER │ 0       │            │ 1  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 1   │ email_address │ TEXT    │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 2   │ created_at    │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 3   │ deleted       │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 4   │ settings      │ TEXT    │ 0       │            │ 0  │ 0      │
└─────┴───────────────┴─────────┴─────────┴────────────┴────┴────────┘

➜  wrangler d1 execute db-enam --command "PRAGMA table_xinfo(users)"

🌀 Mapping SQL input into an array of statements
🌀 Parsing 1 statements
✔ Select an account › a single plum floating in perfume
🌀 Executing on db-enam (c89db32e-83f4-4e62-8cd7-7c8f97659029):
🚣 Executed 1 commands in 0.20019398629665375ms
┌─────┬───────────────┬─────────┬─────────┬────────────┬────┬────────┐
│ cid │ name          │ type    │ notnull │ dflt_value │ pk │ hidden │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 0   │ user_id       │ INTEGER │ 0       │            │ 1  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 1   │ email_address │ TEXT    │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 2   │ created_at    │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 3   │ deleted       │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 4   │ settings      │ TEXT    │ 0       │            │ 0  │ 0      │
└─────┴───────────────┴─────────┴─────────┴────────────┴────┴────────┘

➜  wrangler d1 execute db-enam --command "PRAGMA table_xinfo(users)"

🌀 Mapping SQL input into an array of statements
🌀 Parsing 1 statements
✔ Select an account › a single plum floating in perfume
🌀 Executing on db-enam (c89db32e-83f4-4e62-8cd7-7c8f97659029):
🚣 Executed 1 commands in 0.20019398629665375ms
┌─────┬───────────────┬─────────┬─────────┬────────────┬────┬────────┐
│ cid │ name          │ type    │ notnull │ dflt_value │ pk │ hidden │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 0   │ user_id       │ INTEGER │ 0       │            │ 1  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 1   │ email_address │ TEXT    │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 2   │ created_at    │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 3   │ deleted       │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 4   │ settings      │ TEXT    │ 0       │            │ 0  │ 0      │
└─────┴───────────────┴─────────┴─────────┴────────────┴────┴────────┘

➜  wrangler d1 execute db-enam --command "PRAGMA table_xinfo(users)"

🌀 Mapping SQL input into an array of statements
🌀 Parsing 1 statements
✔ Select an account › a single plum floating in perfume
🌀 Executing on db-enam (c89db32e-83f4-4e62-8cd7-7c8f97659029):
🚣 Executed 1 commands in 0.20019398629665375ms
┌─────┬───────────────┬─────────┬─────────┬────────────┬────┬────────┐
│ cid │ name          │ type    │ notnull │ dflt_value │ pk │ hidden │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 0   │ user_id       │ INTEGER │ 0       │            │ 1  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 1   │ email_address │ TEXT    │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 2   │ created_at    │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 3   │ deleted       │ INTEGER │ 0       │            │ 0  │ 0      │
├─────┼───────────────┼─────────┼─────────┼────────────┼────┼────────┤
│ 4   │ settings      │ TEXT    │ 0       │            │ 0  │ 0      │
└─────┴───────────────┴─────────┴─────────┴────────────┴────┴────────┘

M

Matt Silverlock•7/7/23, 12:21 AM

I'm off for the night, but I'd strongly recommend:

Going through the D1 tutorial first if you haven't. If you're also new to SvelteKit, then you're just adding friction: https://developers.cloudflare.com/d1/get-started/
Once you have that working, deploy a basic Svelte site to Pages: https://developers.cloudflare.com/pages/framework-guides/deploy-a-svelte-site/
Then you can combine it all

C

cris•7/7/23, 12:22 AM

Okay, thanks for your help!

C

cris•7/7/23, 1:02 AM

FIY -- https://temp-d1demo-ts.pages.dev/api/users it's working for me. I made a fresh repo with JSDoc and it didn't work. Then I made a fresh repo and created the sveltekit app with TypeScript and it worked. Thanks so much for this. It's really interesting to get this all combined and working.

V

vassbence•7/7/23, 8:18 AM

How many regions will D1 replicate to? Regional level?

V

vassbence•7/7/23, 10:31 AM

any predictions? i'm evaluating using D1 or Dynamo and if the replication is similar to like a 3-4 region deployment with a singler writer and N read replicas I might aswell just stick with Dynamo

V

vassbence•7/7/23, 10:31 AM

They surely wont put replicas in every PoP

Vvassbence any predictions? i'm evaluating using D1 or Dynamo and if the replication is sim...

M

Matt Silverlock•7/7/23, 2:33 PM

Keep in mind that each Dynamo replica adds to your storage costs + adds additional replication costs. D1 will not for the 'default' global replication.

M

Matt Silverlock•7/7/23, 2:33 PM

(I like a lot of what Dynamo did, but replication stacks up fast: https://aws.amazon.com/dynamodb/pricing/on-demand/)

M

Matt Silverlock•7/7/23, 2:37 PM

https://silverlock-d1-remix-svelte-e.cloudflare-docs-7ou.pages.dev/d1/examples/

what framework should we add next? The Pages docs cover the whole list, but (IMO) having specific examples you can copy+paste to get started for D1 == easier

U

Unsmart•7/7/23, 2:39 PM

I assume D1 will only be single region writer? Or will it be like Dynamo where you can write to the primary node in each region?

UUnsmart I assume D1 will only be single region writer? Or will it be like Dynamo where y...

M

Matt Silverlock•7/7/23, 2:43 PM

Single writer: we think (and others like PlanetScale & Google's Spanner team also share this view) that multiple writers introduces too many consistency issues for user-facing apps. DynamoDB makes a bunch of trade-offs (and they are welcome to!) that we don't think are the right ones for user-facing apps / cause you to have to think about consistency a lot in your app design.

MMatt Silverlock https://silverlock-d1-remix-svelte-e.cloudflare-docs-7ou.pages.dev/d1/examples/ ...

V

vassbence•7/7/23, 3:08 PM

Next.js definitely

Vvassbence Next.js definitely

M

Matt Silverlock•7/7/23, 3:09 PM

https://github.com/timothymiller/t4-app

GitHub

GitHub - timothymiller/t4-app: Type-Safe, Full-Stack Starter Kit fo...

Type-Safe, Full-Stack Starter Kit for React Native + Web. - GitHub - timothymiller/t4-app: Type-Safe, Full-Stack Starter Kit for React Native + Web.

V

vassbence•7/7/23, 3:09 PM

In terms of scale do you think you will have an upper cap or it's planned to support even terrabytes in db size?

Vvassbence In terms of scale do you think you will have an upper cap or it's planned to sup...

M

Matt Silverlock•7/7/23, 3:10 PM

I don't think single databases with terabytes of storage are likely, but I also don't think they make sense elsewhere. A Postgres database or MySQL database with > 100GB will start to see scaling and query performance issues. If you need a single DB with terabytes of storage there's likely an application architecture issue.

C

Colin•7/7/23, 3:15 PM

Colin

C

Colin•7/7/23, 3:23 PM

Sorry if this has been asked before.. I am new to a discord and to cloudflare. The documentation says it is not stable and thus@not ready for production. What other options are available for a cloudflare-distributed web-app that requires a basic database?

CColin Sorry if this has been asked before.. I am new to a discord and to cloudflare. ...

M

Matt Silverlock•7/7/23, 3:34 PM

https://developers.cloudflare.com/workers/databases/connecting-to-databases/

MMatt Silverlock https://github.com/timothymiller/t4-app 🙂

J

James•7/7/23, 3:43 PM

from the looks of things, that repo only interacts with D1 in a worker script, not Next.js, so I would probably avoid using that as an example for Next.js personally

JJames from the looks of things, that repo only interacts with D1 in a worker script, n...

M

Matt Silverlock•7/7/23, 3:48 PM

https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-site/#use-bindings-in-your-nextjs-application is the generic example, that applies to KV/R2/D1

M

Matt Silverlock•7/7/23, 3:48 PM

This isn't to say we won't add a pointed D1 example, but there is a Next.js example

MMatt Silverlock https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-site/#u...

J

James•7/7/23, 3:52 PM

I know - I was the one who updated that

I'm just saying that the example you linked to the person who said Next.js doesn't actually use it in Next.js

MMatt Silverlock This is _definitely_ not related to Chaika’s issue. I think you have a typo in y...

C

ChaikaOP•7/7/23, 3:53 PM

Did you ever get a chance to check into the broken d1 db 972558fd-cadf-4bec-8b0b-5a140d7d51fb? I don't mean to bother you, not critical, I'm duplicating all my writes/etc to Planetscale for this purpose, but it's still dead as a doornail, seems to go between the states of being completely dead/unreachable (Gateway timeouts) and instant d1 errors

D1_ERROR: Error: Internal error while starting up D1 storage caused object to be reset

D1_ERROR: Error: Internal error while starting up D1 storage caused object to be reset

D1_ERROR: Error: Internal error while starting up D1 storage caused object to be reset

D1_ERROR: Error: Internal error while starting up D1 storage caused object to be reset, weird to me since it's just all of a sudden it died, I'm doing a lot of batched writes/updates to it, maybe maxed out some invisible limit, it's on the experimental backend

My D1 DB running the new experimental backend has stopped working entirely, any clue on why? All Que

Similar Threads

Similar Threads

Similar Threads